Papers with psychological tests
Define, Evaluate, and Improve Task-Oriented Cognitive Capabilities for Instruction Generation Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent work examines the cognitive capabilities of language models through psychological tests designed for humans. |
| Approach: | They propose to use human-like cognitive capabilities to evaluate language models . they propose to augment language models with better listeners to improve their performance . |
| Outcome: | The proposed method boosts language models with better models of the listener and improves them. |
Measure Children’s Mindreading Ability with Machine Reading (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing scoring models do not take the features of the stories and video clips into account when scoring, which will reduce the accuracy of the models. |
| Approach: | They propose to leverage the features extracted from stories and videos related to the questions being asked during the children’s mindreading evaluation. |
| Outcome: | The proposed framework agrees well with human experts on scores produced by the models. |